Papers with Generating free-text rationales
REV: Information-Theoretic Evaluation of Free-Text Rationales (2023.acl-long)
Copied to clipboard
| Challenge: | Existing metrics for rationale evaluation focus on the association between the rationale and a label, whereas REV is more sensitive to new information in free-text rationales. |
| Approach: | They propose a metric called REV to quantify the amount of new, label-relevant information in a rationale beyond the information already available in the input or the label. |
| Outcome: | The proposed metric is consistent with human judgments on rationale evaluations and provides more sensitive measurements of new information in free-text rationales. |
Persuasiveness of Generated Free-Text Rationales in Subjective Decisions: A Case Study on Pairwise Argument Ranking (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on generating free-text rationales has focused on tasks where there is an expected factual ground truth. |
| Approach: | They analyze generated free-text rationales in tasks with subjective answers . they find open-source LLMs generate highly persuasive rationale models . |
| Outcome: | The proposed model outperforms closed-source models in pairwise argument ranking, a highly subjective task with potential for debate assistance. |